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FOREWORD 

This  report  is  an  outgrowth  of  the  work  of  the 
Defense  Science  Board  Task  Force  on  Test  and  Evaluation, 
and  the  checklists  herein  have  been  derived  from  the 
study  of  past  major  weapon  system  programs. 

The  T&E  expert  in  reading  this  volume  will  find 
many  precepts  which  will  strike  him  as  being  too  obvious 
to  be  included  in  checklists  of  this  type.  These  items 
are  included  because  examples  were  found  where  even  the 
obvious  has  been  neglected,  not  because  of  incompetence 
or  lack  of  personal  dedication  by  the  people  in  charge 
of  the  program,  but  because  of  financial  and  temporal 
pressures  which  forced  competent  managers  to  compromise 
on  their  principles.  It  is  hoped  that  the  inclusion  of 
the  obvious  will  prevent  repetition  of  the  serious  errors 
which  have  been  made  in  the  past  when  such  political, 
economic  and  temporal  pressures  have  forced  project 
managers  to  depart  from  the  rules  of  sound  engineering 
practices. 

In  the  long  run,  taking  short  cuts  during  T&E  to 
save  time  and  money  will  result  in  significant  increases 
in  the  overall  costs  of  the  programs  and  in  the  delay  of 
the  delivery  of  the  corresponding  weapon  systems  to  the 
combatant  forces. 
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T&E  GUIDELINES  FOR  MISSILE  WEAPON  SYSTEMS 


The  checklist  items  presented  here  are  specifically  applicable  to 
missile  testing  and  evaluation.  It  is  suggested  that  the  user  of  this 
volume  also  refer  to  the  Report  of  the  Defense  Science  Board  on  Test  and 
Evaluation  which  contains  general  checklist  items  also  applicable  to  this 
system  T&E  program.  The  checklist  items  presented  here  are  organized  into 
time  phases  of  the  acquisition  process  oriented  to  the  DSARC  cycle. 

The  checklists  cover  various  aspects  of  the  major  activities  that 
should  be  underway  during  a  given  time  period.  Hence,  a  checklist  might 
cover  the  (1)  evaluation  of  work  that  occurred  in  the  previous  phase, 

(2)  conduct  of  tests  planned  in  the  previous  phase  and  executed  in  the 
subject  phase,  and  (3)  plans  and  other  preparatory  actions  for  test  sche¬ 
dules  to  be  conducted  in  a  subsequent  phase.  For  reasons  such  as  this, 
items  on  some  subjects,  such  as  development  teat  plans,  may  appear  in  more 
than  one  phase.  In  addition,  since  the  Services  and  the  DSARC  have  flexi¬ 
bility  in  deciding  how  rapidly  to  progress  in  the  validation  phase,  there 
may  be  cases  where  the  Request  for  Proposals  (RFPs),  proposal  evaluations, 
source  selections,  or  contract  negotiations  may  occur  after  the  DSARC 
approves  full-scale  development  instead  of  before.  For  this  reason,  it 
is  recommended  that  previous  checklists  in  the  Validation  Phase  be  re¬ 
viewed  when  entering  the  Full-Scale  Engineering  Development  Phase.  The 
following  are  the  phases  used  in  this  report. 

CONCEPTUAL  PHASE 

The  checklist  items  in  this  phase  are  for  guidance  in  evaluating  T&E 
activities  during  the  Conceptual  Phase  of  weapon  systems  acquisition. 

This  phase  (often  research  and  exploratory  development)  precedes  the  first 
DSARC  milestone  and  is  focused  on  the  development  of  a  weapon  system  con¬ 
cept  that  offers  high  prospects  of  satisfying  an  identified  military  need. 

Although  not  called  for  in  DoD  Directive  5000.1  specifically,  the 
objectives  of  this  phase  should  be: 
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1.  To  verify  that  there  Is  a  military  need  for  the  proposed 
system. 

2.  To  demonstrate  that  there  is  a  sound  physical  basis  for  a  new 
weapon  system. 

3.  To  formulate  a  concept,  based  on  demonstrated  physical 
phenomena,  for  satisfying  the  military  need. 

4.  To  show  that  the  proposed  solution  is  superior  to  its  com¬ 
petitors  in  terms  of  potential  effectiveness,  probability  of 
success,  probable  cost,  Impact  on  the  U.S.  military  posture, 
and  development  risks. 

5.  To  analyze  the  technology  outlook  and  the  military  need  to 
show  that  it  is  better  to  start  advanced  development  now 
rather  than  to  wait  for  future  technological  improvements. 

6.  To  identify  the  key  risk  areas  and  critical  issues  that  need 
to  be  resolved  before  full-scale  development  is  initiated. 

The  most  important  product  of  this  phase  is  the  Development  Concept 
Paper  (DCP)  or  its  equivalent.  The  DCP  defines  program  issues,  including 
special  logistics  problems,  program  objectives,  program  plans,  performance 
parameters,  areas  of  major  risk,  system  alternatives,  and  acquisition 
strategy. 


VALIDATION  PHASE 

The  checklist  items  in  this  phase  are  for  guidance  in  conducting  T&E 
during  the  Validation  Phase  (the  time  between  when  the  DSARC  recommends 
approval  of  the  DCP  for  the  first  time  and  when  the  DSARC  recommends  full- 
scale  development  of  the  system). 

While  these  objectives  are  not  spelled  out  in  the  DoD  Directive  5000.1, 
the  objectives  of  the  Validation  Phase  should  be  to  confirm: 

1.  The  need  for  the  selected  system  in  consideration  of  the  threat, 
system  alternatives,  special  logistics  needs,  estimates  of 
development  costs,  preliminary  estimates  of  life  cycle  costs 
and  potential  benefits  in  context  with  overall  DoD  strategy  and 
fiscal  guidance. 

2.  The  validity  of  the  operational  concept. 

3.  That  development  risks  have  been  identified  and  solutions  are 
in  hand. 

4.  Realism  of  the  plan  for  full-scale  development. 


2 


In  the  pursuit  of  the  above  objectives,  it  is  likely  that  advanced 
development  T&E  will  be  conducted  to  resolve  issues.  In  some  cases,  an 
RFP  for  full-scale  engineering  development  will  be  prepared,  proposals 
will  be  received  and  evaluated,  and  contracts  negotiated  in  preparation 
for  seeking  DSARC  approval  for  the  next  phase.  Therefore,  some  checklist 
items  are  included  to  help  ensure  that  this  work  properly  reflects  the 
T&E  interests  in  this  and  subsequent  phases.  For  example,  the  RFP  must 
include  adequate  guidance  to  ensure  that  sufficient  resources  and  time  are 
available  so  that  engineering  effort  can  properly  support  the  initial  DT&E 
with  hardware,  software,  technical  data,  and  training. 

The  primary  emphasis  of  OSD/T&E  activities  is  with  items  3  and  4  above. 
Special  attention  should  be  given  to  the  planning  of  IOT&E  activity  as  it 
is  Incorporated  in  the  engineering  development  contract  as  well  as  the 
DT&E  associated  with  addressing  the  critical  issues  and  areas  of  major 
risk  identified  in  the  DCP. 


FULL-SCALE  ENGINEERING  DEVELOPMENT  PHASE 

The  checklist  items  contained  in  this  phase  are  for  guidance  in 
conducting  T&E  during  the  Full-Scale  Engineering  Development  Phase.  This 
Includes  the  major  DT&E  and  the  IOT&E  conducted  prior  to  the  major  pro¬ 
duction  decision.  By  this  time,  the  weapon  system  is  well-defined  and  is 
becoming  a  unique  item  and,  hence,  sound  judgment  must  be  applied  in  using 
these  checklist  items. 

To  enter  the  Engineering  Development  Phase,  the  DSARC  will  have: 

•  Confirmed  the  need  in  consideration  of  the  threat,  alternatives, 
logistic  needs,  cost,  and  benefits. 

•  Identified  development  risks. 

•  Confirmed  the  realism  of  the  development  plan. 

Given  the  above,  the  primary  objectives  of  the  DT&E  should  be  to: 

1.  Demonstrate  that  the  engineering  and  design  and  development 
process  is  complete  and  that  the  design  risks  have  been  mini¬ 
mized  (the  system  is  ready  for  production) . 

2.  Demonstrate  that  the  system  will  meet  specifications. 


The  primary  objectives  of  the  IOT&E  should  be  to: 

3.  Assess  operational  suitability  and  effectiveness. 

4.  Validate  organizational  and  employment  concepts. 

5.  Determine  training  and  logistic  requirements. 

In  addition,  the  validity  of  the  plan  for  the  remainder  of  the  program 
must  be  confirmed  by  the  DSARC  before  substantial  production/deployment 
will  be  recommended  to  the  Secretary  of  Defense. 

The  level  of  OSD/T&E  activity  is  highest  during  this  phase.  The 
IOT&E  plan  must  be  designed,  the  tests  conducted,  and  the  data  analyzed 
to  evaluate  the  inputs  associated  with  the  primary  objectives.  These 
tests  should  not  be  conducted  until  the  primary  objectives  of  the  DT&E 
have  been  met.  Thus,  OSD/T&E  activity  is  required  to  assess  that  the  DT&E 
major  milestone — the  system  is  ready  for  production — has  been  achieved. 
Close  monitoring  of  the  T&E  Service  activity  is  required  during  the  latter 
stages  of  this  phase. 

SUBSTANTIAL  PRODUCTION/DEPLOYMENT  PHASE 

The  checklist  items  contained  in  this  phase  are  for  guidance  in  con¬ 
ducting  T&E  after  the  substantial  production  decision  has  been  made  by  the 
DSARC.  This  includes  DT&E  and  follow-on  OT&E  to  be  conducted  on  early 
production  items. 

To  enter  the  Production/Deployment  Phase,  the  DSARC  will  have  re¬ 
viewed  the  program  to  confirm: 

•  The  need  for  the  system. 

•  A  practical  engineering  design  with  adequate  consideration  of 
production  and  logistic  problems  is  complete. 

•  All  technical  uncertainties  have  been  resolved  and  opera¬ 
tional  suitability  has  been  determined  by  T&E. 

•  The  realism  of  the  plan. 

The  primary  objective  of  the  DT&E  in  this  phase  should  be  to: 

1.  Verify  that  the  production  system  meets  specifications. 

The  primary  objectives  of  the  follow-on  OT&E  should  be  to: 
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2.  Validate  the  operational  suitability  and  effectiveness. 

3.  Optimize  organization  and  doctrine. 

A.  Validate  training  and  logistic  requirements. 

At  this  point,  the  OSD/T&E  activity  is  similar  to  that  in  the 
previous  phase;  however,  much  of  the  testing  is  verification  that  the 
production  system  performance  is  as  expected.  Hence,  most  of  the  items 
in  the  previous  phase  are  appropriate  to  this  phase,  especially  those 
related  to  OT&E. 


i 
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I.  CONCEPTUAL  PHASE 


During  this  phase  the  program  is  being  conceived  and  the  DCP 
prepared.  The  test  and  evaluation  checklist  covers  consideration 
following: 

1.  Weapon  System  Interfaces 

2.  Number  of  Test  Missiles 

3.  T&E  Gap 

4.  Feasibility  Tests 

5.  Evaluation  of  Conceptual  and  Validation  Tests 

6.  Joint  Testing  Plans 

7.  Nuclear  Weapons  Effects 

8.  Test  Facilities  and  Instrumentation  Requirements 


Preceding  page  blank 


1.  WEAPON  SYSTEM  INTERFACES 


Consider  significant  weapon  system  interfaces,  their  test  require¬ 
ments  and  probable  costs  at  the  outset  of  the  Conceptual  Phase. 

Ensure  that  the  program  plan  assembled  before  DSARC  I  includes  an 
understanding  of  the  basic  test  criteria  and  broad  test  plans  for  the  whole 
program.  These  should  include  time  and  costs  required  for  testing  system 
interfaces.  Initial  program  plans  sometimes  have  not  anticipated  the  need 
for  adequate  testing.  For  example,  at  least  one  program  tested  the  flight 
article  for  an  extended  period  of  time  before  actually  launching  it  from 
the  operationally-configured  launcher.  The  operationally-configured  laun¬ 
cher/missile  system  revealed  unexpected  interactions  which  were  so  differ¬ 
ent  from  the  original  design  intent  that  modifications  were  required  and 
much  of  the  testing  had  to  be  repeated.  In  another  case,  late  selection  of 
a  new  avionics  suit  for  the  carrier/aircraft  made  it  impossible  to  define 
the  missile/aircraft  interface  at  the  start  of  the  fixed  price  contracts. 

This  led  to  several  design  interactions  to  make  the  interface  work  correctly. 

2.  NUMBER  OF  TEST  MISSILES 

Ensure  that  there  is  sufficient  time  and  a  sufficient  number  of  test 
articles  to  support  the  program  through  its  various  phases. 

Compare  the  program  requirements  with  past  missile  programs  of  generic 
similarity.  If  there  is  substantial  difference,  then  adequate  justifica¬ 
tion  should  be  provided.  The  DT&E  period  on  many  programs  has  had  to  be 
extended  as  much  as  50  percent.  Has  the  new  program  provided  reasonable 
time  to  anal  ze  difficulties  and  failures  before  the  following  test?  Two 
of  the  major  problems  in  test  programs  have  been: 

(a)  Insufficient  sample  size,  not  only  of  complete  missiles  but 
also  of  subsystems,  components  and  piece  parts,  to  test  all 
modes  that  stress  the  system  throughout  the  performance  en¬ 
velope  and  to  develop  reliability. 
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(b)  Some  significant  problems  found  during  development  have  not 
been  adequately  corrected  prior  to  production,  apparently 
because  of  insufficient  time  or  the  unavailability  of  test 
articles. 

There  must  be  sufficient  time  and  resources  put  into  the  early  planning 
for  the  operational  tests,  particularly  the  IOT&E  phase,  because  the  produc¬ 
tion  and  deployment  decision  is  planned  to  be  made  based  upon  IOT&E  results. 
Correspondingly,  later  in  the  program,  one  should  not  lose  sight  of  the 
continuing  need  for  adequate  test  articles.  In  the  past,  for  example,  there 
have  been  cases  of  unanticipated  multi-azimuth  launch  requirements  which 
necessitated  additional  testing. 

Requalification  of  piece  parts  and  assemblies  has  also  frequently  been 
required  for  many  reasons.  This  results  in  a  need  for  additional  testing. 
Again,  in  many  cases  there  has  been  insufficient  time  allowance  in  a  test 
program  to  permit  proper  correction  of  items  which  have  caused  failures. 

Experienc  has  shown  that  there  has  been  a  marked  tendency  to  be  over 
optimistic  in  planning  time  and  resources  for  test. 

There  have  been  circumstances  where,  for  various  reasons,  the  technol¬ 
ogy  has  had  to  be  pushed  rapidly;  the  higher  risks  involved  have  required 
more  thorough  testing  and/or  greater  resources.  These  programs,  typically, 
have  been  extended  to  longer  times. 

3.  T&E  GAP 

Encourage  actions  to  eliminate  or  minimize  any  test  and  evaluation  gap. 

A  test  and  evaluation  gap  has  been  experienced  in  some  missile  programs 
between  the  t.ine  when  testing  with  R&D  hardware  was  completed  and  the  time 
when  follow-on  operational  suitability  testing  was  initiated  with  production 
hardware.  This  gap  has  been  as  long  as  2  years.  The  gap  was  generally 
caused  because  all  available  R&D  hardware  was  expended  in  DT&E  and  early 
operational  testing.  Following  this  a  production  decision  was  made  and 
production  hardware  was  not  available  for  many  months.  The  initial  pro¬ 
duction  hardware  was  first  needed  for  several  months  of  unit  training  of 
the  operational  organization.  Finally,  the  production  system  operational 
suitability  flight  tests  were  begun.  By  that  time,  in  one  case,  the 
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production  program  had  reached  maximum  rate. 

The  problems  associated  with  the  test  and  evaluation  gap  are: 

(a)  Deficiencies  that  show  up  in  the  operational  hardware  may  be 
very  costly  to  correct  if  the  production  program  has  been 
accelerated. 

(b)  Time  is  lost  during  which  no  system  flight  experience  is 
gained  to  further  shake-down  the  system  or  to  learn  to  use 
the  system  tactically. 

(c)  Fixes  developed  late  in  the  R&D  program  cannot  be  flight 
tested  in  a  timely  manner- 

(d)  The  production  team  phases  down  after  building  R&D  items 
and  later  builds  up  again  for  the  production  effort.  This 
is  a  costly  process. 

(e)  The  military  personnel  who  gained  experience  in  the  10T&E 
tend  to  be  lost  to  the  follow-on  OT&E  due  to  the  time  factor 
and  the  personnel  turnover  problem. 

Some  ways  in  which  a  program  can  be  designed  to  minimize  the  T&E  gap 
effects  are  to: 

(a)  Plan  use  of  production  engineering  inputs  early  in  the 
initial  R&D  design  to  facilitate  later  production. 

(b)  Plan  to  continue  IOT&E  to  bridge  the  gap  between  completion  of 
the  usual  IOT&E  performed  with  R&D  hardware  and  the  start  of 
follow-on  OT&E  on  full  production  hardware.  A  quandary  in  initial 
operational  suitability  testing  is  that,  on  one  hand,  Congress  is 
reluctant  to  approve  production  until  IOT&E  is  complete,  and  on 
the  other  hand,  IOT&E  may  not  be  meaningful  unless  it  is  conducted 
with  production-like  missiles.  Therefore,  unless  low  rate  pilot 
production  hardware  is  available  for  IOT&E,  the  evaluators  must 
carefully  define  the  hardware  that  is  used  to  conduct  the  IOT&E. 

In  some  cases,  handbuilt  R&D  type  hardware  used  for  IOT&E  may 

be  of  different  quality  material  than  could  be  expected  from 
mass  production. 


4.  FEASIBILITY  TESTS 

Ensure  experimental  test  evidence  is  available  to  indicate  the 
feasibility  of  the  concept  and  the  availability  of  the  technology 
for  the  system  development. 


Experimental  evidence  should  be  available  to  provide  for  an  evaluation 
of  the  proposed  concept  and  the  alternative  approaches  for  accomplishing  the 
prime  mission  of  the  new  system.  For  example,  if  propulsion  state-of-the 
art  is  inadequate  for  the  job,  a  program  in  the  propulsion  area  should  be 
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conducted  in  the  Definition  Phase.  There  is  ample  evidence  where  well 
executed  experimental  test  programs  to  prove  feasibility  were  the  key  to 
new  and  successful  system  developments.  Where  this  is  not  done,  there  is 
considerable  danger  of  program  failure  in  pressing  the  state-of-the-art 
very  far. 


5.  EVALUATION  OF  CONCEPTUAL  AND  VALIDATION  TESTS 

Results  of  tests  conducted  during  the  conceptual  and  the  validation 
phases,  which  most  likely  have  been  conducted  as  avionics  brassboard, 
breadboard,  or  as  modified  existing  hardware,  should  be  evaluated 
with  special  attention. 

Tests  should  be  evaluated  with  attention  to  items  such  as  the  following 

•  Scaling  laws  may  invalidate  the  findings  or  introduce 
new  technology  problems.  Model  tests  will  not 
necessarily  scale  up  or  down  as  expected  in  missiles 
launched  underwater,  in  RV  motion  dynamics  or  in  solid 
rocket  propulsion  units. 

•  The  laboratory-type  environment  in  which  the  hardware 
was  tested  may  preclude  the  generation  of  data  needed 
to  validate  that  the  concept  and  technology  approach 
will  be  applicable  to  an  operational  environment. 

•  The  tests  may  not  include  signals  and  noise  sources 
representative  of  those  that  might  be  expected  in  an 
operational  environment.  Nuclear  background  is  a  typical 
example. 


6.  JOINT  TESTING  PLANS 

When  a  new  missile  development  program  requires  joint  testing  during 
OT&E,  the  test  plan  should  include  the  type  of  tests  and  resources 
required  from  other  activities  and  services. 

Several  weapon  systems  designed  for  defense  penetration  or  defense 
suppression  have  had  inadequate  testing  and  evaluation  in  relation  to  their 
mission  objective. 

In  general,  the  T&E  plan  should,  as  far  as  possible,  include  offense/ 
defense  engagements  representative  of  those  in  which  the  new  system  is 
expected  to  operate.  Many  programs  have  not  given  full  attention  to  this. 
Offense/defense  testing  may  be  addressed  in  several  phases,  such  as: 
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(a)  Testing  against  the  best  simulation  of  the  assumed  threat 
which  can  be  made  available,  either  in  the  field  or  in  a 
laboratory. 

(b)  Testing  against  advanced  U.S.  technology  which  may  be 
representative  of  a  potential  threat. 

(c)  Testing  against  electronic  countermeasures  must  be  investi¬ 
gated.  In  the  conceptual  phase  the  matter  of  countermeasures 
must  be  considered  to  ensure  that  the  development  is  sound. 

During  the  development,  countermeasures  are  a  secondary 
consideration;  however,  after  the  new  system  is  developed, 

it  is  necessary  to  ensure  that  it  can  be  used  in  the  presence 
of  countermeasures  by  the  use  of  technical  features  of  the 
hardware/software  or  by  alternate  operating  modes  or  different 
tactics.  Therefore,  later  phases  of  OT&E  testing  should  in¬ 
clude  appropriate  electronic  countermeasure  aspects. 

In  summary,  there  is  considerable  evidence  that  weapon  systems  have 
been  less  satisfactory  under  combat  conditions  than  earlier  testing  promised. 
Had  test  plans  been  more  realistic  with  better  field  simulations,  the  results 
would  have  been  more  meaningful.  This  in  turn  may  demand  joint  Service 
involvements. 


7.  NUCLEAR  WEAPONS  EFFECTS 

The  subject  of  nuclear  weapons  effects  should  be  addressed 
in  the  DCP  when  relevant. 

In  some  systems,  inadequate  attention  has  been  given  to  the  fact  that 
they  are  being  designed  for  possible  operation  in  a  nuclear  environment. 
Early  design  consideration  will  prevent  the  incremental  time  and  expense  in 
attempting  late  remedial  action.  Experience  has  shown  the  importance  of 
giving  early  attention  to  nuclear  weapons  effects  and  vulnerability.  Plans 
must  recognize  that  all  agencies  involved  in  nuclear  weapon  development 
and  vulnerability  (e.g.,  AEC  and  DNA)  must  have  early  participation. 

8.  TEST  FACILITIES  AND  INSTRUMENTATION  REQUIREMENTS 

Before  DSARC  I  the  test  facilities  and  instrumentation  require¬ 
ments  to  conduct  tests  should  be  generally  identified  along  with 
a  tentative  schedule  of  test  activities. 
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The  capability  of  the  test  range  and  the  adequacy  of  the  facilities 
and  instrumentation  should  be  verified;  it  is  also  recommended  that  alterna¬ 
tive  approaches  be  examined  (e.g.,  ranges)  and  the  need  for  instrumentation 
improvements  or  changes  be  identified  early  in  the  program.  Of  prime  impor¬ 
tance  in  missile  flight  testing  are  the  constraints  that  may  be  placed  on  the 
test  because  of  range  and  instrumentation  limitations.  If  range  and  instru¬ 
mentation  factors  are  found  to  cast  significant  doubt  on  the  meaningfulness 
of  the  test  data  because  of  a  lack  of  operational  realism,  steps  necessary  to 
assure  meaningful  data  should  be  identified  and  planned  before  the  plans  are 
included  in  the  DCP. 

Targets  for  short-range  missiles  warrant  specific  mention;  some  missile 
systems  can't  be  adequately  tested  without  targets  that  realistically  simulate 
the  threat.  In  some  cases,  considerable  lead  time  is  required  to  provide  the 
required  target  support;  hence,  target  requirements  to  support  the  tests  must 
be  identified  early. 
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II.  VALIDATION  PHASE 


During  this  phase  Che  program  is  defined  in  detail.  The  major  efforts 
are  to  conduct  tests  and  evaluation  of  any  issues  raised  in  the  DCP  and  to 
plan  the  test  and  evaluation  efforts  for  the  remainder  of  the  program  with 
emphasis  on  full-scale  development  and  IOT&E.  The  test  and  evaluation 
checklist  covers  consideration  of: 

1.  Establish  Test  Criteria 

2.  Human  Factors 

3.  Instrumentation  Diagnostic  Capability  and  Compatibility 

4.  Provisions  for  Test  Failures 

5.  Integrated  Test  Plan 

6.  Test  and  Evaluation  Requirements 

7.  Personnel  Training  Plans 

8.  T&E  Reporting  Format 

9.  Program-to-Program  Crosstalk 

10.  Status  of  T&E  Offices 

11.  Measurement  of  Actual  Environments 

12.  Thoroughness  of  Laboratory  Testing 

13.  Contract  Form 

14.  Government  Test  Support  Commitments 

15.  Participation  of  Operational  Command 

The  reader  should  also  review  the  checklist  items  in  the  previous  phase 
since  many  of  them  will  be  applicable  during  this  phase. 


Preceding  page  blank 
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ESTABLISH  TEST  CRITERIA 


i . 


By  the  end  of  the  validation  phase,  test  criteria  should  be  established 
so  that  there  Is  no  question  as  to  what  will  constitute  a  successful 
test  and  what  performance  Is  expected. 

There  are  many  examples  where  the  objectives  of  a  given  test  were  not 
adequately  defined  or  the  criteria  for  success  delineated  clearly  in  advance. 
To  a  high  degree,  this  reduces  or  even  invalidates  the  value  of  the  test 
process.  Repeat  testing  necessitated  by  such  actions  is  not  uncommon  and  is 
expensive . 

The  test  plans  must  clearly  define  the  primary  and  secondary  objectives 
of  each  test,  the  test  environment,  the  performance  points  to  be  tested,  the 
instrumentation  needs,  the  data  collection  plan,  and  the  data  reduction 
requirements. 


2 .  HUMAN  FACTORS 

Ensure  that  the  test  plan  includes  adequate  demonstration  of  human 
factors  consideration 


At  an  appropriate  time  in  the  concept  definition  or  the  development 
phase,  the  plan  should  include  consideration  of  the  human  factor  concepts 
that  will  be  involved  in  the  operational  system.  Questions  of  safety, 
comfort,  effectiveness  of  man-machine  interfaces,  reaction  time,  performance 
under  stress,  fatigue,  the  number  and  skill  levels  of  the  personnel  required 
and  other  related  factors  must  be  examined.  The  inclusion  of  human  factor 
requirements  in  early  testing  is  extremely  important.  This  should  result  in 
better  performance,  and  a  reduction  in  subsequent  redesign  and  retrofits 
caused  by  the  introduction  of  the  operator  into  the  system. 

For  example,  retargeting  or  selection  of  alternate  weapons  under  battle 
conditions  may  be  limited  by  human  factors.  Early  test  can  bring  out  defi¬ 
ciencies  which  may  be  more  readily  corrected  at  that  time  in  the  program. 

The  maintainability  of  any  system  also  is  clearly  of  great  importance.  Many 
systems  have  failed  to  give  early  consideration  to  maintainability  and 
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operability  from  the  human  factors  standpoint  with  resulting  cost  and  schedule 
implications  as  the  program  nears  operational  status. 

3.  INSTRUMENTATION  DIAGNOSTIC  CAPABILITY  AND  COMPATIBILITY 

Instrumentation  design  with  adequate  diagnostic  capability  and 
compatibility  in  both  DT&E  and  IOT&E  phases  is  essential. 

There  are  many  examples  where  instrumentation  has  been  inadequate  to 
determine  the  actual  cause  of  missile  failures.  This  has  been  evident  with 
both  DT&E  and  IOT&E  hardware.  Much  of  this  difficulty  can  be  avoided  if 
planning  insists  not  only  on  adequate  instrumentation  provisions,  but  also  on 
a  degree  of  compatibility  between  DT&E  and  IOT&E  instrumentation.  The  com¬ 
patibility  implies  that  minimum  disturbance  of  the  operational  mode  of  the 
system  must  be  the  aim  in  all  cases. 

The  instrumentation  package  configuration  should  be  fixed  early  in  the 
design  phase.  For  this  reason,  OT&E  instrumentation  requirements  must  be 
specified  early  in  the  definition  program  and  the  proposed  design  critically 
reviewed  by  the  OT&E  people. 

4.  PROVISIONS  FOR  TEST  FAILURES 

DT&E  and  OT&E  plans  should  make  provisions  for  the  occurrence  of 
failures. 


DT&E  and  OT&E  test  plans  should  include  time  and  resources  necessary 
for  investigating  test  failures  and  for  eliminating  the  cause  of  failure 
before  another  test  flight  takes  place.  As  a  general  rule  no  major  tests 
should  be  repeated  until  the  cause  of  the  preceeding  failure  is  understood 
and  corrective  action  has  been  implemented.  Where  this  is  not  possible, 
the  next  flight  should  have  instrumentation  configured  to  pinpoint  the  cause 
of  prior  failure. 

Test  failures  have  occurred  which  were  not  diagnosed  before  further 
flight  tests  were  made  and  the  failures  repeated.  Careful  diagnosis  and 
effective  remedial  action  prior  to  retest  is  the  proper  procedure  rather 
than  permitting  subsequent  flight  tests  assuming  that  the  same  failure  will 
not  recur. 
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A  percentage  of  the  total  tests  (sorties,  runs,  trials,  experiments) 
should  be  allowed  for  retesting. 


5.  INTEGRATED  TEST  PLAN 

Assure  an  integrated  system  test  plan  that  pre-establishes  mile¬ 
stones  and  go  ils  for  easy  measurement  of  program  progress  at  a 
later  time. 


In  on  R&D  program  some  failures  may  be  expected,  but  the  plan  should 
be  structured  to  generally  ensure  that: 

(a)  Repetitive  failure  will  be  held  at  an  absolute  minimum 
and  each  substantive  failure  will  be  analyzed  and  the 
cause  corrected  before  subsequent  flight  tests. 

(b)  The  schedule  will  accommodate  problems. 

(c)  There  is  a  clear  statement  of  objectives  for  each  test. 

(d)  The  number  of  tests  that  are  expected  to  yield  an  answer 
have  been  identified. 

(e)  Qualification  testing  of  components  and  subsystems  should 
occur  at  the  earliest  possible  time  in  the  program.  Where 
existing  technology  is  involved,  qualification  can  usually 
occur  prior  to  the  onset  of  flight  testing.  Where  advanced 
technology  is  involved,  qualification  should  be  introduced 
just  as  soon  as  practicable  in  the  development  flight  test 
program — but  in  all  cases  before  production  of  operational 
hardware  is  undertaken.  Component  or  subsystem  problems 
that  are  detected  at  the  full  system  flight  test  level  are 
extremely  costly. 

(f)  Clear,  well-defined  milestones  for  review  and  commitment 
to  the  next  test  phase  have  been  defined. 

Almost  without  exception,  every  missile  program  has  experienced  flight 
test  failures  or  anomalies.  The  degree  to  which  corrective  action  has  been 
taken  prior  additional  flight  tests  has  varied  extremely  among  various 
programs.  The  motivation  for  continuing  flight  test  without  adequate  fail¬ 
ure  diagnosis  and  corrective  action  is  often  associated  with: 

(a)  The  desire  to  claim  that  the  program  has  completed  its  major 
milestones. 

(b)  Failure  to  clearly  identify  test  objectives. 

(c)  A  superficial  conclusion  that  the  failure  was  random. 

(d)  Flying  of  unqualified  components  which  should  have  been 
qualified  before  flight. 
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6. 


TEST  AND  EVALUATION  REQUIREMENTS 


V 

$ 


Ensure  that  the  test  and  evaluation  program  requirements  are  firm 

before  approving  an  R&D  test  program. 

Many  missile  programs  have  suffered  severe  cost  impacts  as  a  result  of 
this  deficiency.  The  test  plan  must  include  provisions  to  adequately  test 
those  portions  of  the  operational  envelope  which  stress  the  system  including 
backup  and  degraded  operational  modes.  In  addition,  those  resources  neces¬ 
sary  for  failure  analysis  and  corrective  action,  along  with  those  necessary 
for  weather  and  appropriate  day/night  testing  as  well  as  those  for  offense/ 
defense  testing  must  be  included. 

In  some  cases,  the  basic  R&D  program  tested  only  the  most  simple  flight 
profile.  Later  more  difficult  profiles  were  used  operationally.  In  other 
cases,  inadequate  attention  was  given  to  testing  the  whole  system  until  too 
late  which  resulted  in  late  discovery  of  integration  problems  and  the  neces¬ 
sity  for  expensive  redesign  and  retesting. 

7.  PERSONNEL  TRAINING  PLANS 

Ensure  that  adequate  training  and  certification  plans  for  test 

personnel  have  been  developed. 

Errors  by  test  personnel  are  usually  expensive  and  often  cloud  the 
reason  for  test  failures.  An  independent  survey*  has  shown  that  lack  of 
availability  of  qualified  personnel  was  the  primary  area  contributing  to 
test  errors  in  similar  new  developments.  Because  of  the  cost  and  inherent 
irreversibility  of  missile  testing,  the  quality  and  adequateness  of  train¬ 
ing  and  certification  plans  for  test  personnel  is  of  critical  importance. 

8.  T&E  REPORTING  FORMAT 

Include  a  T&E  reporting  format  in  the  program  plan. 


The  Role  of  Testing  in  Achieving  Aerospace  Systems  Effectiveness,  A  report 
prepared  by  the  AIAA  Technical  Committee  on  Systems  Effectiveness  and 
Safety,  January  1973. 
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Attention  must  be  given  to  the  reporting  format  in  order  to  provide  a 
consistent  basis  for  test  evaluation  throughout  the  program  life  cycle. 
Changes  in  the  reporting  methods  for  test  configurations  and  results  should 
be  carefully  examined  to  ensure  that  rapid  and  accurate  reconstruction  of 
this  data  is  possible  throughout  the  program  life  cycle.  It  is  extremely 
important  to  clearly  identify  the  cause  of  failure  and  to  determine  if  it  is 
random  or  general.  This  can  usually  only  be  accomplished  if  accurate  cor¬ 
relation  of  the  system  test  history  is  possible. 

Some  missile  programs  have  used  T&E  reporting  methods  which  were  not 
adequate  to  clearly  identify  incipient  anomalies  or  even  past  failure  modes. 
In  other  programs  test  reporting  systems  were  changed  with  resulting  degra¬ 
dation  in  the  value  of  trend  data  or  an  inability  to  quickly  track  and 
correct  generic  causes  of  failure. 

9.  PROGRAM-TO-PROGRAM  CROSSTALK 

Encourage  program-to-program  T&E  cross  talk. 

Test  and  evaluation  problems  and  their  solutions  on  one  program  provide 
a  valuable  index  of  lessons  learned  and  techniques  for  problem  resolution  on 
other  programs.  This  is  especially  useful  in  programs  employing  similar 
technologies.  A  strong,  intentional  and  well-organized  effort  to  increase 
cross  fertilization  would  prevent  unnecessary  re-identification  of  the 
problems  and  the  cost  of  re-learning  their  solutions. 

As  an  example,  hypergolic  propellant  usage  matured  on  one  program  can 
be  transferred  with  real  advantage  to  guide  another  program  using  similar 
technologies.  The  same  is  true  for  structures,  guidance,  payload,  integra¬ 
tion,  dynamic  response,  environmental  influence  and  many  other  missile  sub¬ 
systems  and  constraints. 

10.  STATUS  OF  T&E  OFFICES 

Ensure  that  Test  and  Evaluation  offices  have  the  same  stature  as 

other  major  elements,  reporting  to  the  program  manager  or  director. 
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It  is  important  that  the  test  and  evaluation  component  of  the  system 
program  office  have  organizational  status  and  authority  equal  to  configura¬ 
tion  management,  program  control,  system  engineering,  etc.  It  is  essential 
that  data  resulting  from  subsystem  and  system  testing  be  surfaced  and  dealt 
with  quickly  and  effectively.  Test  organizations  which  operate  subordinate 
to  other  program  office  groups  tend  to  develop  the  bias  of  that  organization 
with  resulting  degradation  in  objectivity  of  the  test  result  analysis.  It 
is  often  the  case  that  the  actual  test  result  is  a  measure  of  the  performance 
of  program  control,  configuration  management,  quality  control,  production 
and  engineering  organizations  and  as  such,  provides  a  critical  measure  of 
the  status  of  system  development  and  operational  use. 

11.  MEASUREMENT  OF  ACTUAL  ENVIRONMENTS 

Thorough  measurements  should  be  made  to  define  and  understand  the 
actual  environment  in  which  the  system  components  must  live  during 
the  captive,  launch  and  in-flight  phases. 

For  instance,  new  ballistic  missile  launch  concepts  may  generate  issues 
to  be  resolved  in  the  validation  phase.  These  concepts  may  range  from 
super-hard  silos  to  highly  mobile,  relatively  soft  launchers.  If  testing  is 
necessary  to  demonstrate  one  or  several  launcher  concepts,  ensure  that  all 
significant  interfaces  of  the  launcher  and  the  environment  are  also  tested. 
For  example,  the  cross  country  mobility  may  be  an  issue  of  an  off-road 
launcher.  A  test  of  the  launcher  concept  should,  of  course,  address  the 
major  issue,  but  the  test  should  look  further  for  other  relevant  informa¬ 
tion  that  might  surface  as  a  result  of  a  good  test.  For  example,  in  address¬ 
ing  the  initial  issue  about  off-road  mobility  it  may  be  observed  that  great 
dust  clouds  cause  considerable  difficulty  in  maintaining  the  operability  of 
the  instrumentation.  This  might  be  a  clue  regarding  reliability  of  a 
sophisticated  missile  guidance  system  that  would  require  special  design 
consideration. 

There  are  also  instances  in  which  the  ground  and  aircraft  launching 
mechanisms  or  air-to-surface  missiles  have  undergone  significant  redesign 
resulting  from  captive  flight  and  operational  launchers  measurements,  thus 
indicating  the  desirability  of  accomplishing  these  tests  as  early  as  is 
reasonable  in  the  program. 
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Prior  to  the  freezing  of  the  production  missile  design,  a  thorough 
inflight  measurement  program  should  be  conducted  so  as  to  provide  informa¬ 
tion  on  the  captive,  launch,  and  in-flight  environments  of  the  system. 

In-flight  environments,  especially  vibration,  temperature,  shock  and  stress 
imposed  during  the  operational  use  of  the  missile,  should  be  measured  in 
flight  tests  and  documents. 

12.  THOROUGHNESS  OF  LABORATORY  TESTING 

Significant  time  and  money  will  be  saved  if  each  component,  each 

subsystem,  and  the  full  system  are  all  tested  as  thoroughly  as 

possible  in  the  laboratory. 

There  are  numerous  instances  in  which  very  expensive  retrofits  have 
had  to  be  made  on  deployed  missile  systems  due  to  component  and  subsystem 
reliability  problems  which  could  have  been  averted  through  a  more  thorough 
test  screening  or  life  testing  procedure.  In  addition  there  are  instances 
in  which  flight  test  failures  could  have  been  averted  by  more  thorough  flight 
dynamic  simulation  in  the  laboratory. 

Whenever  field  testing  is  expensive  compared  with  laboratory  and 
simulation  testing  (as  in  missile  or  aircraft/missile  flight  testing),  such 
testing  should  be  conducted  primarily  for  verification  of  design  parameters 
or  design  performance,  rather  than  to  see  whether  or  not  a  particular  component 
or  subsystem  will  work.  The  full  system  test  should  be  simulated  insofar  as 
possible  in  advance  of  the  flight  test.  In  the  case  of  a  tactical  missile, 
for  example,  captive  flight  tests,  as  well  as  ground  and  computer  simula¬ 
tions,  should  be  conducted. 

Other  related  items  are  III-8  on  test  program/range  safety  dry  runs  and 
III-2  on  plans  and  use  of  test  simulations  and  dry  runs. 

13.  CONTRACT  FORM 

The  contract  form  can  be  extremely  important  to  the  T&E  aspects. 

In  one  program  the  contract  gave  the  contractor  full  authority  to 
determine  the  number  of  test  missiles,  and  in  another  the  contract  incentive 
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resulted  in  the  contractor  concentrating  tests  on  one  optimum  profile  to 
satisfy  the  incentive  instead  of  developing  the  performance  throughout 
important  areas  of  the  envelope. 

In  the  Validation  Phase  the  contracting  strategy  is  usually  established 
in  the  RFP  and  the  subsequent  evaluation  and  source  selection  process. 

From  the  T&E  standpoint,  the  contract  should: 

(a)  Permit  early  user  and  evaluator  participation. 

(b)  Establish  incentives  only  after  careful  consideration 
and  resolution  of  possible  undesirable  aspects  from  the 
T&E  standpoint.  The  incentives  should  not  be  based  on 
extreme  corners  of  the  theoretical  performance  envelope 
unless  there  is  a  high  operational  payoff.  On  the  other 
hand,  the  incentive  for  criteria  should  not  constrain  the 
developer  from  exploring  the  likely  operational  performance 
envelope. 

(c)  Facilitate  engineering  changes  resulting  from  knowledge, 
gained  during  the  test  program  and  from  threat  changes. 


14.  GOVERNMENT  TEST  SUPPORT  COMMITMENTS 

Require  T&E  office  coordination  for  contracts  which  propose  to 
make  government  test  support  commitments. 


There  are  missile  programs  in  which  the  government  committed  facility 
support  to  a  fixed  price  contract  with  an  "on-demand"  provision.  The 
result  of  this  was  that  the  priorities  for  facility  utilization  were  not 
consistent  with  national  defense  priorities  in  order  to  prevent  default 
on  the  part  of  the  government. 

If  there  are  GFE  and  other  government  commitments  in  the  proposed  con¬ 
tract,  be  concerned  about  the  following: 

(a)  Can  the  gear  with  required  performance  be  available  for  test 
when  required? 

(b)  Can  government  supported  facilities  provide  the  T&E  assistance 
required  at  the  time  needed?  If  not,  is  it  reasonable  to 
construct  the  required  facilities  (test  range,  instrumenta¬ 
tion,  building,  etc.)?  If  not,  what  alternatives  are  avail¬ 
able? 

(c)  Avoid  contract  terms  on  fixed  price  contracts  that  vaguely 
commit  the  government.  Do  not  include  "government  support 
as  required"  or  "test  facilities  will  be  made  available 
when  needed." 
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(d)  Provide  test  milestones  with  which  to  gauge  progress 
and  peg  decision  points  to  test  progress  and  not  to 
calendar  dates; 

(e)  Provide  test  resources  (R&D,  prototypes,  pilot  production, 
etc.)  for  continuous  testing  activities  until  the  initial 
substantial  production  hardware  is  available. 


15.  PARTICIPATION  OF  OPERATIONAL  COMMAND 

It  is  imperative  that  the  operational  command  actively  participate 
in  the  DT&E  phase  to  ensure  that  the  user  needs  are  represented  in 
the  development  of  the  system. 

Where  user  participation  in  DT&E  has  been  inadequate,  additional 
problems  have  been  discovered  when  the  program  progressed  to  the  operational 
test  phase.  Some  deficiencies  discovered  late,  such  as  human  factors 
problems,  technical  manual  errors,  and  design  inadequacies,  could  have  been 
much  more  easily  corrected  if  they  had  been  found  early  in  the  DT&E  by  user 
participants . 

Initially,  the  operational  test  command  should  plan  an  advisor  role 
during  the  feasibility  and  engineering  testing,  and  finally  take  over  leader¬ 
ship  in  the  conduct  of  the  operational  testing  program.  This  user  partici¬ 
pation  in  DT&E  should  facilitate  the  necessary  communication  and  interaction 
between  the  development  and  the  operational  commands — especially  needed 
during  the  DT&E  and  IOT&E  phases. 
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III.  FULL  SCALE  ENGINEERING  DEVELOPMENT  PHASE 


In  this  phase  the  full-scale  development  and  the  IOT&E  will  be  planned 
in  greater  detail  and  conducted.  The  test  and  evaluation  checklist  includes 
consideration  of: 

1.  Production  Philosophy  and  Techniques 

2.  Operational  Flight  Profiles 

3.  Failure  Isolation  and  Responsive  Action 

4.  Responsive  Actions  for  Test  Failures 

5.  Plan  Tests  of  Whole  System 

6.  Determination  of  Component  Configuration 

7.  Testing  of  Software 

8.  Range  Safety  Dry  Runs 

9.  Assemblies/Subsystems  Special  Requirements 

10.  Review  of  ASM  Test  Position  Fixes 

11.  Operator  Limitations 

12.  Test  Simulations  and  Dry  Runs 

13.  Component  Performance  Records 

14.  Tracking  of  Test  Data 

15.  Updating  of  IOT&E  Planning 

16.  Instrumentation  Provisions  in  Production  Missiles 

17.  Constraints  on  Missile  Operator 

18.  Problem  Fixes  Before  Production 

19.  Flight  Tests  Representative  of  Operations 

The  reader  should  also  review  the  checklist  items  in  the  previous  phases 
since  many  of  them  will  be  applicable  in  this  phase. 
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1.  PRODUCTION  PHILOSOPHY  AND  TECHNIQUES 


Encourage  that  production  philosophy  and  production  techniques 
be  brought  Into  an  early  phase  of  the  design  process  for  R&D 
hardware  to  the  maximum  extent  practical. 

There  are  many  missile  programs  in  which  the  components  were  not 
qualified  until  the  missile  was  well  into  production.  This  situation 
led  to  a  circumstance  in  which  IOT&E  test  results  have  less  than  maximum 
value  and  OT&E  does  not  commence  until  large  numbers  of  units  are  deployed. 

An  intimate  interaction  between  production  engineers  and  design 
engineers,  including  test  personnel,  should  be  established  early  in  the 
program.  This  process  will  tend  to  minimize  the  changes  between  the  R&D 
test  hardware  and  the  production  units;  hence,  there  is  less  probability 
that  new  problems  will  show  up  in  the  follow-on  T&E  tests  of  the  produced 
system.  Some  programs  have  done  this  splendidly  and  saved  much  time  and 
money  in  moving  from  development  to  production. 

2.  OPERATIONAL  FLIGHT  PROFILES 

Tests  should  be  conducted  to  evaluate  all  planned  operational 
flight  profiles  and  all  primary  and  back-up  degraded  operating 
modes. 


The  profiles  need  to  be  evaluated  to  ensure  that  no  unpredicated 
problems  occur.  For  example,  some  air  launched  missiles  can  be  programmed 
to  hit  targets  to  the  side  or  the  rear  as  well  as  to  the  front,  using  high 
altitude  or  low  altitude  flight  profiles.  This  results  in  a  large  number 
of  flight  profile  options.  The  testing  should  at  least  cover  those  planned 
operational  profiles  which  stress  the  system. 

The  backup  degraded  operating  modes  need  to  be  tested  because  aircraft 
avionics  reliability  problems  and  battle  damage  may  cause  the  primary  mode 
to  fail.  For  instance,  a  carrier  may  have  several  navigation  systems  such 
as  inertial,  doppler,  or  bomb/nav  radar  to  provide  position  data  for  missile 
inputs.  All  systems  might  be  used  in  the  primary  operating  mode.  In  an 
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operational  environment,  it  is  not  unlikely  that  some  launches  might  be  made 
with  only  one  of  these  systems  remaining  in  operation.  Therefore,  these 
backup,  degraded  operating  modes  should  be  tested  to  determine  their  charac¬ 
teristics  and  capability. 

3.  FAILURE  ISOLATION  AND  RESPONSIVE  ACTION 

Does  the  system  test  plan  provide  for  adequate  instrumentation  so 
that  missile  failures  can  be  isolated  and  fixed  before  the  next 
flight .  . . .  . 

If  failures  occur,  has  sufficient  subsystem  testing  been  planned  so 
that  the  underlying  cause  of  the  failures  can  be  properly  identified  (random 
failures  seldom  occur)  within  the  C&C,  the  propulsion,  subsystems,  etc.? 

After  the  cause  of  a  failure  has  been  identified  and  fixed,  has  it  been 
adequately  planned  to  be  demonstrated  through  subsystem  testing  that  the 
cause  has  been  removed  and  that  additional  failures  from  that  source  are 
unlikely? 

As  a  general  rule,  do  not  allow  system  testing  to  continue  after  a 
failure  has  occurred  unless  the  above  actions  have  been  taken. 

U,  RESPONSIVE  ACTIONS  FOR  TEST  FAILURES 

Encourage  a  closed  loop  reporting  and  resolution  process  which 
assures  that  each  test  failure  at  every  level  is  closed  out  by 
appropriate  action,  i.e.,  redesign,  procurement,  retest,  etc. 

Sometimes  there  is  reluctance  by  the  design  organization  to  accept  that 
their  part  failed  and  to  quickly  correct  design  deficiencies  discovered  in 
test  programs.  Tight  control,  by  the  SPO,  to  correct  faults  discovered  in 
the  test  program  will  help  ensure  that  the  next  test  does  not  fail. 


5.  PLAN  TESTS  OF  WHOLE  SYSTEM 

Plan  tests  of  the  whole  system  including  proper  phasing  of  the 
platform  and  supporting  gear,  the  launcher,  the  missile,  and 
the  user's  participation. 
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There  have  been  missile  development  programs  where  Important  subsystems 
have  not  been  Included  In  the  test  program  in  a  timely  fashion.  In  a 
tactical  program,  a  non-ope rational  launcher  was  used  for  testing  for  many 
months,  when  the  operational  launcher  was  finally  used,  redesign  and  retest 
was  necessary  because  of  the  different  environmental  conditions.  The  R&D 
tests  should  be  conducted  on  the  whole  system  as  early  as  practical. 

In  other  programs,  the  user  participation  was  either  too  late  or  not 
sufficiently  active.  The  operational  evaluation  unit  personnel  should 
participate  in  the  tests  from  the  beginning  and  they  should  act  both  as 
active  critics  and  as  contributors. 

6.  DETERMINATION  OF  COMPONENT  CONFIGURATION 

Conditions  and  component  configuration  during  development  tests 

should  be  determined  by  the  primary  objectives  of  that  test. 

Whenever  a  non-operational  configuration  is  dictated  by  early  test 
requirements,  tests  should  not  be  challenged  by  the  fact  that  configuration 
is  not  operational.  For  example,  if  in  the  development  of  a  short-range 
surface-to-surface  missile  it  is  found  that  the  test  missiles  tend 
to  hit  the  ground  early,  it  may  be  desirable,  for  a  variety  of  reasons,  to 
fire  the  missile  from  an  elevated  platform  or  across  a  valley  before  the 
problems  of  early  missile  dropoff  are  satisfactorily  solved.  On  the  other 
hand,  demonstration  and  acceptance  tests,  as  well  as  tests  intended  to 
evaluate  performance  under  operational  conditions,  should  always  be  con¬ 
ducted  under  conditions  as  close  to  those  anticipated  in  operations  as 
possible. 

Where  tests  are  run  with  substitute  parts,  be  sure  procedures  are 
established  to  record  the  fact  and  ensure  that  necessary  retesting  is  done 
with  the  correct  components.  When  testing  is  delayed  because  of  the  non¬ 
availability  of  critical  sub-system  components,  off-the-shelf  interim 
components  may  be  used  as  substitutes  until  the  proper  components  are 
available.  As  long  as  the  off-the-shelf  components  can  function  acceptably 
within  a  defined  range  of  interest,  the  rest  of  the  system  can  be  tested, 
thereby  facilitating  the  progress  of  the  test  program.  Clearly,  this  cannot 
be  continued  indefinitely,  but  it  should  serve  to  reduce  the  hindering 
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influence  of  long  lead  time  components  that  are  not  available  on  time. 
Selected  tests  may  have  to  be  repeated  when  the  proper  component  is 
available. 

In  one  system,  an  autopilot  was  used  in  place  of  the  planned  inertial 
guidance  system  until  the  latter  was  ready  for  incorporation  into  the  system. 
This  substitution  permitted  testing  on  the  launcher,  propulsion,  and  general 
flight  characteristics. 

7.  TESTING  OF  SOFTWARE 

Test  and  evaluation  should  ensure  that  software  products  are 
tested  appropriately  during  each  phase. 

Software  has  often  been  developed  more  as  an  add-on  than  as  an  integral 
part  of  the  overall  system.  Software  requirements  need  the  same  considera¬ 
tion  as  hardware  requirements  in  the  Validation  Phases.  Usual  practices 
often  do  not  sufficiently  provide  for  testing  the  software  subsystem  concept. 
Often  the  facilities  available  to  contractors  for  software  development  and 
verification  are  critical  to  schedule  and  cost. 

8.  RANGE  SAFETY  DRY  RUNS 

Ensure  the  test  plan  includes  adequate  test  program/range  safety 
dry  runs. 


The  government  test  ranges  have  to  provide  facilities  to  safely  test 
many  different  projects.  All  test  conducted  have  a  range  safety  officer 
responsible  for  stopping  any  test  that  exceeds  predetermined  limiting  condi¬ 
tions.  A  test  program  manager  should  be  able  to  ensure  that  the  range 
safety  equipment  and  people  used  on  a  wet  run  will  be  the  same  as  the  team 
used  on  the  dry  runs.  Programs  have  experienced  numerous  aborts  and  even 
destruction  of  good  missiles  because  of  problems  in  the  range  safety/test 
interface.  These  problems  usually  develop  because  of  incomplete  dry  runs, 
range  safety  communications  problems,  radar  track  losses  or  other  dropouts 
due  to  equipment  changes  after  the  dry  runs.  Complete  dry  runs  and  range 
cooperation,  in  holding  the  range  safety  teams  together  for  a  given  project, 
will  help  to  minimize  the  problems. 

B 
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9.  ASSEMBLIES/SUBSYSTEMS  SPECIAL  REQUIREMENTS 

Assemblies  and  subsystems  that  may  require  special  attention. 

There  will  be  selected  subsystems  or  assemblies  that  usually  require 
special  attention. 

Seekers  and  tracking  devices  especially  for  automatic  systems  should 
receive  extensive  laboratory  testing  with  use  times  and  environments 
applied  as  realistically  as  possible. 

Propulsion  subsystems  including  feed  lines  and  tankage  should  be 
dynamically  tested  at  all  angles  of  elevation  or  positions  from  the  hori¬ 
zon  which  total  vehicle  is  expected  to  see  in  the  operational  environ¬ 
ment  . 

From  the  onset  of  the  program  special  attention  should  be  given  to 
connectors  and  their  related  hardware.  The  design  or  selection  of 
connectors  should  be  made  with  their  ultimate  use  in  mind.  Environmental 
considerations  as  well  as  connect-disconnect  frequencies  have  proven 
extremely  important. 

Lanyard  assemblies  should  have  extensive  laboratory  testing.  Pull 
forces  should  be  examined  in  all  angles  of  the  pull  cone  and  these  re¬ 
sults  should  subsequently  become  pull  test  criteria  for  the  operational 
system. 

Safing,  arming,  fuzing  and  other  ordnance  devices  should  have  com¬ 
plete  laboratory  work  prior  to  installation  in  the  flight  article.  It  is 
also  suggested  that  alternate  concepts  should  be  considered.  Testing 
should  be  accomplished  for  both  the  preflight  and  flight  environment. 
Batch  or  lot  testing  should  be  required  on  all  operational  hardware. 

These  items  are  included  in  the  checklist  because  they  have  posed 
difficult  and  repeated  problems  on  almost  all  missile  systems.  Dealing 
effectively  with  this  type  of  hardware  early  in  the  program  will  be  cost 
effective  and  may  prevent  major  problems  from  developing  in  the  "all-up 
systems  tests." 
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10.  REVIEW  OF  AIR-TO-SURFACE  MISSILE  TEST  POSITION  FIXES 


Review  the  final  position  fix  planned  before  launching  ASMs. 

There  are  Instances  In  which  the  operational  test  of  air  launched 
missiles  utilized  artificial  position  fixes  just  prior  to  missile  launch. 
Optimistic  data  results,  unless  the  position  firing  test  procedures  are 
similar  to  those  used  on  operational  missiles.  System  accuracy  for 
inertial  guided  air-launched  missiles  will  be  most  dependent  upon  posi¬ 
tion,  velocity,  and  heading  data  provided  by  the  carrier.  If  the  test 
program  uses  beacons,  radar  reflectors  or  other  cooperative  accurate  fix 
features  in  the  near  vicinity  of  the  target,  the  accuracy  results  will 
be  better  than  could  be  expected  on  an  operational  mission.  An  opera¬ 
tional  carrier  may  have  to  plan  to  make  the  last  fix  on  a  prominent 
feature  at  a  considerable  distance  from  the  target  due  to  operational 
considerations. 

11.  OPERATOR  LIMITATIONS 

Ensure  operator  limitations  are  included  in  the  tests. 

Most  tactical  missiles,  especially  those  used  in  close  support, 
require  visual  acquisition  of  the  target  by  the  missile  operator  and/ 
or  an  air/ground  controller.  Thus,  the  system  performance  is  very  much 
dependent  upon  the  operator's  eye  (or  some  visual  aid).  The  ability 
to  acquire  the  target  must  not  be  assumed  trivial  and  must  be  tested 
in  operationally  realistic  conditions  to  determine  to  what  extent  this 
factor  may  limit  the  capability  of  the  total  missile  system. 

In  a  normal  launch  sequence  of  an  E-0  type  missile,  the  operator 
(or  pilot,  in  the  case  of  single  place  aircraft)  is  required  to  (a)  locate 
and  acquire  the  target  visually,  (b)  determine  that  what  he  sees  is  in  fact, 
a  real  target,  (c)  decide  whether  it  lies  within  the  size,  contrast,  and 
range  envelope  of  the  weapon,  (d)  re-acquire  the  target  on  his  cockpit 
display  by  means  of  the  weapon's  seeker,  (e)  verify  that  the  seeker  has 
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achieved  "lock"  on  the  proper  target,  (f)  launch  the  weapon,  and  (g)  man¬ 
euver  away  from  the  target  area.  This  sequence  of  activities  must  be 
conducted  in  a  dynamic  (and  very  likely  hostile)  environment;  the  effect 
is  to  make  system  performance  very  much  dependent  upon  the  operator's 
ability. 

This  item  is  related  to  Item  1-4,  ensure  that  the  concept  is  feasi¬ 
ble. 


12.  TEST  SIMULATIONS  AND  DRY  RUNS 

Plan  and  use  test  simulations  and  dry  runs. 

Dry  runs  should  be  conducted  for  each  new  phase  of  testing.  Simula¬ 
tion  and  other  laboratory  or  ground  testing  should  be  conducted  to  pre¬ 
dict  the  specific  test  outcome.  The  "wet  run"  test  should  finally  be  run 
to  verify  the  test  objectives.  Evaluation  of  the  simulation  versus  the 
actual  test  results  will  help  to  refine  the  understanding  of  the  system. 

Ground  simulation  of  the  electronics  system,  the  flight  control  and 
hydraulics  system,  and  the  other  systems  can  be  most  helpful  in  solving 
functional,  design,  safety,  maintainability  and  reliability  problems. 

Complete  system  simulation  of  the  planned  test  can  greatly  increase 
the  value  of  overall  testing  effort  by  fostering  a  more  complete  apprecia¬ 
tion  of  how  the  system  really  operates. 

Other  related  items  are  11-12  on  laboratory  simulation  testing  and 
III-8  on  test  program/range  safety  dry  runs. 

13.  COMPONENT  PERFORMANCE  RECORDS 

Keep  performance  records  on  components. 

There  are  many  examples  in  missiles  programs  which  have  required 
stock  sweeps  that  are  associated  with  flight  failures  and  aging  testing 
programs.  These  stock  sweeps  can  require  component  lot  identification, 
manufacturing  time  interval,  batch  processing  lot  identification,  etc. 
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When  developing,  testing  and  evaluating  the  various  subsystems  (and 
systems)  of  missile  weapon  systems,  each  component  of  the  systems  should  be 
numbered  and  a  performance  history  kept  which  allows  an  analysis  of  that 
component's  performance,  with  respect  to  reliability,  maintainability, 
availability,  etc. 

14.  TRACKING  OF  TEST  DATA 

Ensure  the  test  program  tracks  data  in  a  readily  usable  manner. 

Reliability  and  performance  evaluations  of  a  missile  system  should 
break  down  the  missile's  activity  into  at  least  the  following  phases: 

•  Pre-launch  including  captive  carry  reliability 

•  Launch 

•  In-flight 

•  Accuracy/fuzing 

Computation  of  the  appropriate  reliability  data  on  this  basis  should 
provide  more  insight  into  test  results  than  more  aggregated,  less  revealing 
indices . 

In  monitoring  test  progress  use  reliability  data  carefully.  In  con¬ 
structing  reliability  data,  moving  averages  for  recent  tests  as  well  as 
overall  averages  should  be  computed,  especially  when  large  numbers  of  tests 
are  involved.  This  helps  to  identify  problems  that  crop  up  late  in  testing. 

Item  II-8  on  T&E  reporting  format  is  related. 

15.  UPDATING  OF  IOT&E  PLANNING 

Periodically  update  military  preliminary  evaluation  (MPE)  and 

IOT&E  planning  during  the  early  R&D  phase. 

Few  missile  system  programs  have  had  adequate  user  participation  with 
the  desirable  continuity  of  personnel  to  minimize  the  problems  of  transition 
from  DT&E  to  OT&E  to  deployment/utilization. 

The  MPEs  may  be  largely  restricted  to  mock-up  exercises  and  user- 
evaluation  participation  in  the  R&D.  One  of  the  user-evaluator's  major 
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activity  should  be  to  use  the  day-by-day  on-the-job  training  experience  with 
the  R&D  system  to  assist  in  preparation  of  the  IOT&E  program  for  user  command, 
support  command,  development  command,  and  contractor  guidance.  A  good  IOT&E 
plan  should  cover  the  allocation  of  manpower  spaces,  assignment  of  personnel, 
personnel  training,  unit  training,  equipment  provisioning  including  technical 
manuals,  ground  support  equipment,  spares  and  missiles,  and  launcher (s). 

The  test  plans  may  include  instrumentation  plans,  ground  tests,  and  flight 
tests  of  important  profies.  The  data  collection,  analysis,  and  reporting 
schemes  must  be  formulated. 

The  practicality  of  finding  the  right  conditions  for  some  of  these  tests 
during  a  reasonable  IOT&E  period  may  require  some  conditions  to  be  simulated 
and  possibly  some  to  be  deferred  as  goals  for  the  follow-on  OT&E.  For 
example,  cold  weather  test  can  only  be  conducted  in  the  northern  hemisphere 
in  winter.  Some  winters  are  mild.  A  wait  until  the  next  winter  for  desired 
cold  weather  is  probably  not  reasonable  from  the  program  standpoint.  Simu¬ 
lation  facilities,  such  as  the  Climatic  Hanger  at  Eglin  AFB,  Florida  can  be 
used  to  provide  reasonable  assurance  of  the  cold  weather  characteristics. 

These  user-evaluators  involved  in  the  MPEs  and  IOT&E  should  also  pre¬ 
pare  the  follow-on  OT&E  plan.  Since  the  follow-on  OT&E  is  intended  to  be 
an  indicator  of  operational  suitability  in  the  hands  of  a  typical  unit,  the 
early  user-evaluators  should  preferably  be  continued  in  the  evaluator  role 
instead  of  in  the  operator/maintenance  activities. 

16.  INSTRUMENTATION  PROVISIONS  IN  PRODUCTION  MISSILES 

Encourage  built-in  instrumentation  provisions  in  production  missiles. 

In  the  more  expensive  missiles  instrumentation  leads  can  probably  be 
built  in  with  little  cost  or  performance  compromise.  This  is  done  in  a  recent 
ASM  program.  With  these  provisions,  instrumentation  can  be  added  to  more 
completely  evaluate  OT&E  tests  and  other  investigative  tests  on  the  opera¬ 
tional  systems  such  as  searches  for  causes  of  troubles  or  in  periodically 
monitoring  life  tests.  Instrumented  missiles,  where  feasible  without 
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compromising  the  operational  character  of  the  hardware,  will  provide  much 
valuable  data  to  help  get  maximum  information  out  of  each  test. 

Item  II-3  on  DT&E  and  IOT&E  instrumentation  is  related. 

17.  CONSTRAINTS  ON  MISSILE  OPERATOR 

Detailed  test  plans  should  be  evaluated  to  determine  that  the  test 
imposed  constraints  on  the  missile  operator  do  not  invalidate  the 
applicability  of  the  data  so  collected. 

The  performance  factor  must  be  a  major  objective  in  any  OT&E  of  the 
missile  weapon  system.  Constraints  such  as  requiring  the  operator  to  per¬ 
form  atypical  functions  (such  as  imposed  by  the  test  to  collect  data)  may 
throw  doubt  on  the  validity  of  the  data  as  it  relates  to  weapon  system 
effectiveness.  On  the  other  hand,  realistic  combat  operator  task  loading 
should  be  included  to  the  extent  possible. 

Item  II-2  relates  to  human  factor  demonstrations  in  test  planning. 

18.  PROBLEM  FIXES  BEFORE  PRODUCTION 

Ensure  operational  suitability  tests  identify  operational  deficiencies 
of  new  systems  quickly  so  that  fixes  can  be  developed  and  tested  before 
large  scale  production. 

Operational  tests  should  be  used  to  identify  problem  areas  associated 
with  the  tactics  of  delivering  the  missile  system  in  realistic  combat  type 
conditions  which  can  significantly  degrade  the  operational  effectiveness  or 
utility  of  the  weapon  system.  For  example,  an  optically  guided  missile 
tracking  subsystem  may  not  function  as  desired  for  all  sun  angle  conditions; 
semi-active  laser  guided  seekers  require  reflected  energy  from  the  target, 
thus  imposing  possible  constraints  on  the  geometry  of  the  designator, 
delivery  system  and  the  target.  Human  factor  problems  associated  with  dis¬ 
plays,  task  loading  and  weapons  delivery  functions  may  restrict  total 
capability  of  the  system. 

These  problem  areas  need  to  be  identified  very  early  in  the  IOT&E  phase 
so  that  possible  engineering  fixes  can  be  studied,  developed  and  tested 
prior  to  large  scale  production  acquisition  to  allow  for  a  more  tactically 
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usable  missile.  Furthermore,  problem  areas  associated  with  human  factors 
can  be  examined  to  determine  special  training  requirements. 

Item  IV-6  is  a  reminder  to  emphasize  fix  testing  not  completed  in  IOT&E. 
Item  IV-7  is  also  related  and  refers  to  OT&E  feed-back  to  acceptance  testing. 

19.  FLIGHT  TESTS  REPRESENTATIVE  OF  OPERATIONS 

Ascertain  that  final  DT&E  system  tests  and  IOT&E  flight  tests  are 

representative  of  operational  flights. 

Some  ballistic  missile  R&D  programs  have  shown  very  high  success  rates 
in  R&D  flight  test;  however,  when  the  early  production  systems  were  deployed, 
they  exhibited  a  number  of  unsatisfactory  characteristics  such  as  poor  alert 
reliability  and  poor  operational  test  flight  reliability.  Be  alert  to  condi¬ 
tions  in  the  T&E  that  may  give  satisfactory  R&D  tests  but  result  in  poor 
operational  experience.  For  example,  in  the  R&D  the  maintenance  equipment 
used  to  support  the  operational  ground  equipment  (OGE)  may  be  more  sophis¬ 
ticated  test  gear  than  is  used  operationally.  All  R&D  missiles  may  be 
launched  using  one  set  of  highly  tuned  OGE.  The  use  of  one  set  or  a  small 
number  of  highly  groomed  ground  systems  may  adversely  affect  the  value  of 
the  relatively  large  number  of  successful  R&D  test  flights. 

Look  for  ways  more  effectiveness  might  result  if  different  or  addi¬ 
tional  things  were  done  in  R&D  and  IOT&E  aimed  at  precluding  the  costly  mods 
to  correct  early  production  deficiencies. 


36 


iV,  SUBSTANTIAL  PRODUCTION /DEPLOYMENT  PHASE 


This  phase  occurs  after  the  DSARC  substantial  production  decision. 
Follow-on  OT&E  will  be  conducted  with  production  hardware.  The  test  and 
evaluation  checklist  includes  consideration  of: 

1.  System  Interfaces  in  Operational  Test 

2.  Realistic  Conditions  for  Operational  Testing 

3.  Testing  of  all  Operational  Modes 

4.  Extension  of  OT&E  for  New  Threats 

5.  "Lead-the-Fleet"  Production  Scheduling 

6.  Test  Fixes  from  IOT&E 

7.  OT&E  Feedback  to  Acceptance  Testing 

The  reader  should  also  review  the  checklist  items  in  the  previous  phases, 
especially  the  last  phase,  since  many  of  these  items  will  be  applicable  during 
this  phase. 
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1.  SYSTEM  INTERFACES  IN  OPERATIONAL  TEST 


Ensure  the  primary  objective  of  an  operational  test  is  to  obtain 

measurements  on  the  overall  performance  of  the  weapon  system  when 

it  is  interfaced  with  those  systems  required  to  operationally  use 

the  weapons  system. 

Some  missile  systems,  such  as  ICBMs,  are  usually  a  complete  system 
developed  by  one  project  office  and  an  operational  test  would  be  difficult 
to  even  try  without  using  the  whole  system,  with  the  possible  exception  of 
part  of  the  C3.  On  the  other  hand,  some  of  the  tactical  systems  must  operate 
from  several  different  carriers  and  some  can  be  used  in  a  direct  mode,  where 
the  target  is  acquired  from  the  carrier  before  missile  launch,  and  some  can 
be  used  in  the  indirect  mode,  where  the  target  is  selected  by  a  second  party 
and  the  missile  acquires  after  launch.  An  operational  test  for  overall 
performance  of  a  tactical  missile  system  used  in  a  direct  fire  mode  should 
include  (as  a  function  of  target  type,  terrain,  weather,  tactics,  etc.): 

•  Readiness  (measured  b;  maintainability,  supportability,  alert 
reliability,  etc.) 

•  Target  acquisition  capability  in  terms  of  range  and  probability 

•  Missile  launch  capability  in  terms  of  range  and  probability 

•  Survivability  of  the  launcher  and  the  target  designator 

•  Total  probability  of  target  kill. 

Testing  of  terminally  guided  missiles  used  in  an  indirect  fire  mode 
must  also  provide  data  on  the  accuracy  of  location  of  the  target  and  the 
capability  to  launch  the  missile  into  the  necessary  basket  in  order  for  the 
terminal  guidance  to  be  operative.  In  addition,  the  communications  between 
the  designator  and  the  launcher  are  important. 


38 


2.  REALISTIC  CONDITIONS  FOR  OPERATIONAL  TESTING 


Ascertain  operational  testing  Is  conducted  under  realistic  combat 
conditions. 


This  means  that  the  offense/defense  battle  needs  to  be  simulated  in  some 
fashion  before  the  evaluation  of  the  weapon  system  can  be  considered  completed. 
Whether  this  exercise  is  conducted  within  a  single  service  (as  in  the  test  of 
a  surface-to-surface  anti-tank  missile  against  tanks)  or  between  services  (as 
in  the  test  of  an  air-to-surface  missile  against  tanks  with  anti-aircraft 
protection) ,  the  plans  for  such  testing  should  be  formulated  as  part  of  the 
system  development  plan. 

Special  attention  should  be  placed  on  the  realistic  environment  to  be 
used  in  the  operational  test,  on  the  vulnerability  of  the  various  elements 
of  the  missile  system  (FAC,  platform,  target  designator,  etc.)  and  on  the 
use  of  models  and  simulators  to  evaluate  the  post  launch  activities  affecting 
target  kills  (probability  of  break  lock  of  the  missile,  reliability,  invalid 
launches,  etc.)  when  the  missile  is  not  launched  during  tests. 

Similarly,  the  OT&E  offense/defense  tests  should  be  performed  in  appro¬ 
priate  weather.  Ballistic  missiles  and  their  ground  systems  should  be  tested 
under  extremes  of  weather.  The  tests  should  include  the  launch  control  center, 
the  communications,  and  the  launcher,  whether  it  be  a  silo,  a  submarine,  or  any 
other.  The  value  of  this  was  demonstrated  by  one  system  that  was  deployed  for 
some  time  before  it  was  discovered,  one  winter  day,  that  a  silo  wouldn't  open 
due  to  an  ice  and  snow  condition.  It  may  also  be  reassuring  to  know  that  a 
missile  had  beeu  successfully  test  launched  through  a  thunderstorm  or  a  heavy 
sea.  The  practicality  of  finding  the  right  conditions  for  some  of  these 
tests  during  a  reasonable  IOT&E  may  tequire  some  conditions  to  be  simulated 
and  possibly  some  to  be  deferred  as  goals  for  the  follow-on  OT&E.  In  summary, 
the  goal  is  that  the  full  production  equipment  should  be  tested  under  appropriate 
offense/defense,  day/night,  weather  conditions. 

Item  1-6  covers  preliminary  test  plans  for  offense/defense  testing. 
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3. 


TESTING  OF  ALL  OPERATIONAL  MODES 


Assure  the  follow-on  OT&E  plan  Includes  tests  of  any  operational 
modes  not  previously  tested  in  IOT&E. 

All  launch  modes.  Included  degraded,  backup  modes,  should  be  tested  in 
the  follow-on  OT&E  because  the  software  interface  with  the  production  hard¬ 
ware  system  should  be  thoroughly  evaluated,  otherwise  small,  easy  to  fix 
problems  might  preclude  launch.  To  do  this  might  cause  extension  of  the 
R&D/IOT&E  phase  because  of  the  operational  flexibility  of  some  ballistic 
missiles,  particularly  those  with  MIRV  options  or  MIRV/PENAID  combinations. 

Item  1 1 1—2  states  that  tests  should  be  conducted  to  evaluate  all 
operational  flight  profiles  and  all  primary  and  back-up  degraded  operating 
modes  in  the  DT&E  and  IOT&E. 

4.  EXTENSION  OF  THE  OT&E  FOR  NEW  THREATS 

Be  alert  to  the  need  to  extend  the  OT&E  if  a  new  threat  shows  up. 


Very  few  missile  programs  perform  any  kind  of  tests  relatable  to 
evaluating  system  performance  against  current  threats,  let  alone  new 
threats.  It  is  important  to  ensure  that  adequate  testing  is  carried  out 
against  current  threats  and  that  the  T&E  function  be  responsive  to  new 
threats,  should  they  be  identified  later. 

The  operational  test  should  be  re-instituted  to  evaluate  system 
adequacy  in  the  face  of  new  threats  encountered  after  the  system  is 
operational  and  the  first  follow-on  OT&E  phase  is  complete.  The  con¬ 
tinuing  test  activities  that  usually  generate  reliability  and  CEP  data 
for  war  plan  purposes,  should  be  considered  also  as  a  source  of  test 
resources  for  testing  responses  to  new  threats. 

5.  "LEAD-THE- FLEET"  PRODUCTION  SCHEDULING 

"Lead-the-Fleet"  missile  scheduling  and  tests  should  be  considered. 

In  almost  every  missile  program  there  are  items  which  have  limited 
shelf  life  or  have  caused  performance  variations  as  a  function  of  age. 
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A  missile's  peacetime  life  will  consist  of  storage  checkouts  and,  for 
ASMs  most  of  their  life  will  be  spent  hanging  on  an  aircraft  on  the  ramp 
plus  several  hundred  hours  of  being  carried  on  an  airplane  in  flight.  Some 
of  the  earliest  pilot  production  missiles  should  oe  devoted  to  "lead  the 
fleet"  for  reliability  and  life  tests.  These  ASMs  should  be  required  to 
accumulate  many  more  ramp  exposure  hours  and  captive  flight  hours  than 
usual  prior  to  launch. 

A  propulsion  life  test  program  should  be  set  up  early  in  the  production 
phase  of  any  rockets.  A  selected  number  of  propulsion  states  from  each 
production  block  should  be  set  aside  for  long-time  storage  under  environ¬ 
mental  conditions  simulating  those  to  which  the  operational  missiles  are 
exposed.  These  motors  would  be  inspected  periodically  as  the  operational 
life  expectancy  is  reached  or  if  propulsion  aging  problems  appear  earlier, 
these  motors  provide  a  test  sample  for  firing  demonstration. 

Safe  and  arming  (SAF)  systems  which  contain  chemical  igniters  should 
similarly  be  set  aside  for  aging  tests.  Any  detonators  used  in  thrust 
termination  blow-out  parts,  explosive  bolts  used  in  stage  separation,  or 
similar  chemical  systems  that  might  deteriorate  should  also  be  set-up  for 
a  life  assurance  monitoring  program. 

In  this  fashion,  T&E  on  production  "lead  the  fleet"  missiles  can 
provide  early  indications  of  problem  areas  and  possibly  preclude  dangerous 
situations  or  stand-downs  of  important  capabilities. 

6.  TEST  FIXES 

Test  fixes  resulting  from  earlier  operational  testing. 

Following  initial  operational  tests  which  identify  problem  areas  in 
missiles,  follow-on  OT&E  should  be  alert  in  these  areas  with  the  primary 
intent  of  investigating  the  adequacy  of  the  fixes  incorporated,  partic¬ 
ularly  if  the  IOT&E  did  not  run  long  enough  to  test  the  fixes.  For  example, 
on  one  program  a  rocket  nozzle  deficiency  showed  up  in  a  ground  qualification 
test  of  a  motor  just  at  the  end  of  the  equivalent  of  IOT&E  testing.  Fixes 
were  developed  and  ground  tests,  but  extensive  full  system  flight  tests  with 
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were  developed  and  ground  tests,  but  extensive  full  system  flight  tests  with 
the  fixed  nozzle  had  to  await  follow-on  OT&E. 

Item  111-18  addresses  the  desirability  of  finding  and  fixing  such 
problems  in  IOT&E.  Also  see  IV-7. 

7.  OT&E  FEEDBACK  TO  ACCEPTANCE  TESTING 

Ensure  OT&E  results  are  quickly  fed  back  to  influence  early  production 

acceptance  testing. 

Production  acceptance  testing  is  probably  the  final  means  the  govern¬ 
ment  will  normally  have  to  ensure  the  product  meets  specifications.  That 
early  acceptance  testing  could  be  influenced  favorably  by  a  quick  feedback 
from  follow-on  OT&E  to  acceptance  testing  is  exemplified  by  a  current  ASM 
program  where  production  has  reached  peak  rates  and  the  OT&E  has  not  been 
completed. 

Items  III-18  and  IV-6  are  related. 
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